seeds=address-1,address-2,address-3
With RHQ 4.8.0, numeric metrics are no longer stored in the RHQ relational database. Metric data is instead stored in RHQ Storage Nodes. While you can run RHQ with a single Storage Node, clustering is at the core of the RHQ Storage Node architecture. Let's first consider some reasons why you may want to run multiple Storage Nodes.
Increase write throughput
Failover/High Availability
Data replication
Distribute CPU load
Distribute load on disk
Distribute load on memory
When you execute rhqctl install with no arguments, the storage node, the server, and the agent will be installed in that order. Prior to installation, if you have already determined that you will run multiple storage nodes, you can avoid some substantial maintenance overhead by deploying the storage nodes prior to installing the server. This section provides steps for setting up a three node cluster prior to installing the server.
This guide assumes each RHQ storage node and RHQ server will run on their own machines. You will be performing a non-default installation of each storage node which means a rhq-storage.properties file will be specified with the --storage-config option for rhqctl install. Please review RHQ Storage Node Installation and the documentation for the --storage-config option for the rhqctl install command if you have not already done so.
Determine the IP addresses of the machines on which storage nodes will be running.
Some storage node settings are shared cluster-wide and must be the same for each node. The following properties (that can appear in rhq-storage.properties) are cluster-wide settings:
Property |
Default Value |
client-port |
9142 |
storage-port |
7100 |
ssl-storage-port |
7101 |
If you specify any of these in a rhq-storage.properties file with --storage-config, then the values must be the same for each storage node.
Use the --storage-config option to specify the seeds property along with any other custom configuration for the storage node. The seeds property should look like,
seeds=address-1,address-2,address-3
where address-1, address-2, and address-3 refer to the IP addresses of storage node's 1 through 3 respectively.
Now install the storage node.
$ rhqctl install --storage --storage-config=rhq-storage.properties
This will install the storage node and the agent but not the server.
Just like we did for the first storage node, create a rhq-storage.properties file that includes a seeds property along with any other custom configuration
seeds=address-1,address-2,address-3
Now install the storage node.
$ rhqctl install --storage --storage-config=rhq-storage.properties
Just like we did for the first and second storage nodes, create a rhq-storage.properties file that includes a seeds property along with any other custom configuration
seeds=address-1,address-2,address-3
Now install the storage node.
$ rhqctl install --storage --storage-config=rhq-storage.properties
On the machine where you will install the RHQ Server, you will have to update <rhq-install-dir>/bin/rhq-server.properties with connection information for the storage nodes. You need to edit the rhq.cassandra.seeds property which should look like,
# A comma-delimited list of seed nodes. Each Cassandra node uses this list as # contact points to find other nodes in the cluster and learn the ring # topology. The RHQ server(s) use this list to connect to the cluster; # therefore, this property MUST BE SET in order for RHQ to talk to Cassandra. # Each entry in the list consists of three fields that are pipe-delimited. The # of an entry is as follows, # # hostname|jmxPort|nativeTransportPort| # rhq.cassandra.seeds=address-1|7299|9142,address-2|7299|9142,address-3|7299|9142
Note that the example uses default values for the jmxPort and nativeTransportPort connection properties (7299 and 9142 respectively). If you specified non-default values with the --storage-config option, be sure to substitute the correct values that you have set in your storage configuration file.
After you finish configuring the server for installation, install with
$ rhqctl install --server --agent
This will install the server and the agent but no storage node.
Let's assume that have a default installation of server, storage node, and agent all running on the same machine. You want to install a second storage node on another machine.
Determine the IP address of the machine on which storage node will be running.
If you used any non-defaults for cluster-wide properties with the first storage node, make certain that the second storage node has the same values.
Use the --storage-config option to specify the seeds property along with any other custom configuration for the storage node. The seeds property should look like,
seeds=address-1
where address-1 is the IP address of the initial storage node that is already installed
Do not include address-2 in the seeds list. Doing so will cause the new node to bootstrap incorrectly and result in critical data loss.
Now install the storage node.
$ rhqctl install --storage --storage-config=rhq-storage.properties
And you are done. There is no need to apply configuration changes to the first storage node or the server. The necessary changes will be applied automatically when the new storage node is auto-imported into inventory.